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Abstract 

This paper considers the consensus problem for a network of nodes with random inter¬ 
actions and sampled-data control actions. We first show that consensus in expectation, in 
mean square, and almost surely are equivalent for a general random network model when 
the inter-sampling interval and network size satisfy a simple relation. The three types of 
consensus are shown to be simultaneously achieved over an independent or a Markovian 
random network defined on an underlying graph with a directed spanning tree. For both 
independent and Markovian random network models, necessary and sufficient conditions for 
mean-square consensus are derived in terms of the spectral radius of the corresponding state 
transition matrix. These conditions are then interpreted as the existence of critical value 
on the inter-sampling interval, below which global mean-square consensus is achieved and 
above which the system diverges in mean-square sense for some initial states. Finally, we 
establish an upper bound on the inter-sampling interval below which almost sure consensus 
is reached, and a lower bound on the inter-sampling interval above which almost sure diver¬ 
gence is reached. Some numerical simulations are given to validate the theoretical results 
and some discussions on the critical value of the inter-sampling intervals for the mean-square 
consensus are provided. 

Keywords: Consensus; Markov chain; sampled-data; random networks 

1 Introduction 

In the traditional consensus algorithm, each node exchanges information with a few neighbors, 
typically given by their relative states, and then updates its own state according to a weighted 
average. It turns out that with suitable (and rather general) connectivity conditions imposed on 
the communication graph, all nodes asymptotically reach an agreement in which the nodes’ initial 
values are encoded [T][^. Various consensus algorithms have been proposed in the literature. The 
most common continuous-time consensus algorithm is given by an ordinary differential equation 
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in terms of the relative states of each agent with respect to its neighboring agents The 

agent state is driven towards the states of its neighbors, so eventually the algorithm ensures 
that the whole network reaches an agreement provided that the network is jointly connected. 
In [^[^, the authors developed discrete-time consensus algorithms. In such an algorithm, each 
agent updates its states as a convex combination of the state of itself and that of its neighboring 
agents. Due to the fact that most algorithms are implemented by a digital device and that 
the communication channels are unreliable and often subject to limited transmission capacity, 
sampled-data consensus algorithms have also been proposed [6j|^. In a sampled-data setting, 
the agent dynamics are continuous and the control input is piecewise continuous. The closed- 
loop system is transformed into discrete-time dynamics and conditions on uniform or nonuniform 
sample periods are critical to ensure consensus. 

Consensus over random networks has drawn much attention since communication networks 
are naturally random. In [Io||ll| , the authors studied distributed average consensus in sen¬ 
sor networks with quantized data and independent, identically distributed (i.i.d.) symmetric 
random topologies. The authors of evaluated the mean-square convergence of consensus 
algorithms with random asymmetric topologies. Mean-square performance for consensus algo¬ 
rithms over i.i.d. random graphs was studied in |13], and the impact of random packet drops 


was investigated in 14 . Recently, the i.i.d. assumption was relaxed in 15,^ to the case where 
the communication graph is modeled by a finite-state Markov chain. Probabilistic consensus 


has also been investigated in the literature. It was shown in 17 that for a random network 
generated by i.i.d. stochastic matrices, almost sure, in probability, and (p > 1) consensus are 


equivalent. In 18 , the authors showed that almost sure convergence is reached for i.i.d. random 
graphs, and Erdos-Renyi random graphs. The analysis was later extended to directed graphs 


and more general random graph processes 19,20 . In 21 , the authors showed that asymptotic 
almost sure consensus over i.i.d. random networks is reached if and only if the graph contains a 
directed spanning tree in expectation. Divergence in random consensus networks has also been 
considered, as representing asymptotic disagreement in social networks. Almost sure divergence 


of consensus algorithms was considered in 22,23 


In this paper, we consider sampled-data consensus problems over random networks. We 
analyze the convergence of the consensus algorithm with a sampled-data controller under two 
random network models. In the first model, each node independently samples its neighbors 
in a random manner over the underlying graph. In the second model, each node samples its 
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neighbors by following a Markov chain. The impact of the sampling intervals on consensus 
convergence and divergence is studied. We consider consensus in expectation, mean-square, 
and almost sure sense. We believe that the models considered in this paper is applicable to 
some applications since they incorporate sampling by digital devices, limited node connections, 
and random interactions imposed by unreliable networks. The main contributions of this paper 
are summarized as follows. For both independent and Markovian random network models, 
necessary and sufficient conditions for mean-square consensus are derived in terms of the spectral 
radius of the corresponding state transition matrix. These conditions can be interpreted as 
critical thresholds on the inter-sampling interval and we show that they can be computed by 
a generalized eigenvalue problem, which can be stated as a quasi-convex optimization problem. 
For each random network model, we obtain an upper bound on the inter-sampling interval below 
which almost sure convergence is reached, and a lower bound on the inter-sampling interval above 
which almost sure divergence is reached. To the best of our knowledge, this is the first time 
that almost sure consensus convergence and divergence are studied for sampled-data systems, 
and also the first time that almost sure divergence is considered for Markovian random graphs. 

The remainder of the paper is organized as follows. Section provides the problem formu¬ 
lation, and introduces the probabilistic consensus notions. Their relations are also discussed. 
Section [^focuses on independent random networks. In this section, we present necessary and/or 
sufficient conditions for expectation consensus, mean-square consensus, almost sure consensus, 
and almost sure divergence. The same problems are addressed under a Markovian network 
in Section Compared with random networks, Markovian networks allow each link to be a 
channel with “memory”. In Section we illustrate our theoretical results through numerical 
simulations. Finally, some concluding remarks are drawn in Section]^ 

Notations: N, C, M and M+ are the sets of nonnegative integers, complex numbers, real 
numbers and positive real numbers, respectively. For x, y G M, x V y and x A y stand for the 
maximum and minimum of x and y respectively. The set of n by n positive semi-definite (positive 
definite) matrices over the field C is denoted as S” (§++)■ For a matrix X = [xi X 2 • • • x^] G 
l^mxn, ||jj-|| j-epregents the spectral norm of X; X* and X' are the Hermitian conjugate and 
the transpose of X respectively. The Kernel of X is defined as Ker(X) = {u G M” : Xv = 0} . 
vec{X) is the vectorization of X, i.e., vec(X) := [xi,X 2 , ■ ■., x(j]^ G (g) denotes a Kronecker 
product of two matrices. If m = n, p{X) and Tr(X) are the spectral radius and the trace of X 
respectively. For vectorization and Kronecker product, the following properties are frequently 
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used in this work: i) vec{ABC) = {C ® yl)vec(i?); ii) {A ® B){C ® D) = {AC) ® (BD), where 
A, B, C and D are matrices of compatible dimensions. For vectors x,y G x _L y is a short 
hand for {x,y) = 0, where (•,•) denotes Euclidean inner product. The indicator function of a 
subset £/ C O is a function 1^ : n —{0,1}, where 1 ^/( 0 ;) = 1 if w G ^ and 1 ^( 0 ;) = 0 if 
u) ^ . The notation cj(-) represents the cr-algebra generated by random variables. Depending 

on the argument, | • | stands for the absolute value of a real number, or the cardinality of a set. 


2 Problem Formulation 


2.1 Sampling and Random Networks 


Consider a network of N nodes indexed in the set V = {1,2,. ..,N}. Each node i holds a value 
Xi(t) G M for t G [0,oo). The evolution of Xi{t) is described by 

Xi{t)=Ui{t), (1) 


where G M is the control input. 


The directed interaction graph G = (V, E) describes underlying information exchange. Here 
E C V X V is an arc set and {j, i) G E means there is a (possibly unreliable) communication link 
from node j to node i. The set of neighbors of node i in the underlying graph G is denoted as 
Ai := {j : {j,i) ^ E}- The Laplacian matrix L := [lij] G associated with G is defined as 

_ j -1, if i / j and {j, i) G E 

i-flmibSE}) A i — j. 

A directed path from node ii to node ii is a sequence of nodes {ii,... ,ii} such that {ij,ij+i) £ E 
for j = 1,..., Z — 1. A directed tree is a directed subgraph of G = (V, E) such that every node 
has exactly one parent, except a single root node with no parents. Therefore, there must exist 
a directed path from the root to every other node. A directed spanning tree is a directed tree 
that contains all the nodes of G. 

Let ^ be associated with G and the set containing all subgraphs of G and {G^ = (V, Efc)}fcgf^ 
be a sequence of random graphs, in which by definition each G^ is a random variable taking 
values in 1^. The Laplacian matrix L{k) := [lij{k)] G associated with G^ is defined as 


Zij (Zc) 


-1, if i / j and (j,z) G Efc 

if * — j. 
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The set of neighbors of node i in denoted as Nii{k) := {j : {j, z) G Efc}. Let the triple P) 

denote the probability space capturing the randomness contained in the random graph sequence, 
where T is the set of all subsets of . Furthermore, we define a filtration Tk = o'(Go, • • •, G^) 
for fc G N. 

We define a sequence of node sampling instants as 0 = to <■■■< tk < tk+i < ■ ■ ■ with 
Tk = tfc+i ~ tk representing the inter-sampling interval. The sampled-data consensus scheme 
associated with the random graph sequence {Gfcjfcgi^ is given by 

Ui{t) = ^ [xj{tk) - Xi{tk)], t£[tk,tk+i)- (2) 

The closed-loop system can then be written in the compact form 


x{tk+i) =[I - TkL{k)\x{tk) := W{k)x{tk) 


(3) 


with W{k) := [wij{k)]. 


Remark 1 In the sampled-data algorithm each node samples its own state at the sampling 
instants node has continuous access to its own state for all t > 0, we can 

introduce the algorithm 


as considered in 


241 


Ui{t)= [xj{tk) - Xi{t)], t£[tk,tk+i), 

The corresponding closed-loop system is then 
x(4+i) =[/-(!- e~'^'^)L{k)]x{tk). 


(4) 


(5) 


By replacing Tk in i) with 1 — e in all the conclusions in this paper for i) throughout 
the paper can thus be readily translated into those for 


2.2 Consensus Metrics 

Define Xuiax{tk) '■= niaxjgv and Xmin(tfc) := xaini^y Xi{tk) and the agreement measure 

X{k) := Xmax(tfc) ~ Xminitk)- We have the following definitions for consensus convergence and 
divergence. 

Definition 1 (i) Algorithm ([^ achieves (global) consensus in expectation if for any initial 

state x{to) G there holds limfc_^oo IE[T(/c)] = 0. 
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(a) Algorithm (§ achieves (global) consensus in mean square if for any initial state x{to) G 
there holds linifc^oo = 0. 

(Hi) Algorithm (© achieves (global) consensus almost surely if for any initial state x(io) £ 
there holds P (limfc_^oo N{k) = 0) = 1. 

(iv) Algorithm Q diverges almost surely if there holds j£(A;) = cx)^ = 1 for any 

initial state x{to) G except for x{to) -L 1- 


2.3 Relations of Consensus Notions 


The following lemma suggests that if the inter-sampling interval is small enough, the consensus 
notations in Definitionare equivalent. 

Lemma 1 Suppose Tk G (O, (N — 1)“^] for all k. Then expectation consensus, mean-square 
consensus, and almost sure consensus are all equivalent for Algorithm (§. 

Proof. We begin with the observation that W{k) is a row stochastic matrix for all A: G N when 
Tfc G (O, (N —1)“^], where a row stochastic matrix means a nonnegative square matrix with 
each row summing to 1. Therefore, 

N N 

Xmax(4+l) = I^i^X Wjj {k)xj (tfc) < I^ay Wjj {k) {xj (4) V Xmax(4)) = Xmax(4), 

i=i i=i 

implying that Xmax(Afe) is non-increasing in k. We show that Xmin(Afc) is non-decreasing in k 
in precisely the same way. The foregoing two observations together suggest that X{k) is non¬ 
increasing in k. Finally, the conclusion follows by showing the following implications: 


(i) Expectation consensus mean-square consensus. Since X{k) is non-increasing, we have 
E[X^(/c)] < X(0)E[X(t)]. By the hypothesis, E[X^(A:)] < T(0)E[jC(A:)] —)• 0 as /c —>• oo. 


(ii) Mean-square consensus almost sure consensus. According to Chebyshev’s inequal¬ 
ity 25 , 

holds for any e > 0. If limfc^oo E[X^(/c)] = 0, consequently limfc_>.oo E (|T(A:)| < e) = 0. As 
a result, there exists a subsequence of {j£(A:)}fcgN that converges to 0 almost surely (^ . 
Since {X{k)}k^^ is non-increasing, lim^^oo 3i{k) = 0 almost surely. 
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(iii) Almost sure consensus => expectation consensus. Since the sequence {X(A:)}fcgN is non¬ 
negative and non-increasing, by the Monotone Convergence Theorem [26| , lim^^oo E[X(A;)] = 

0 . 


□ 


Remark 2 In '211, the equivalence of IP consensus, consensus in probability, and almost sure 


consensus was obtained over a random network generated by i.i.d. stochastic matrices. In 
Lemma [I| we show that this equivalence holds regardless of the type of random process the row 
stochastic matrices are generated by. The equivalence relation follows from the monotonicity of 


3 Independent Random Networks 

In this section, we investigate sampled-data consensus when the random graph is obtained 
by each node independently sampling its neighbors in a random manner over G. Regarding the 
connectivity of the underlying graph G, we adopt the following assumption: 

(Al) The underlying graph G has a directed spanning tree. 

We also impose the following assumption. 

(A2) The random variables Uc) £ E, /c G N, are i.i.d. Bernoulli with mean g > 0. 

The techniques developed in this section also apply when q = q{i) is a function of node index i. 

In order to simplify the notation used in the derivation of the results through this section, 
we also make the following assumption. 

(A3) Let Tfc = r* for all k with r* > 0. 

When each node samples its neighbors as Assumption (A2) describes, {L{k)}ken are i.i.d. 
random variables, whose randomness originates from the primitive random variables 
We denote the sample space of L{k) by .if := where M = and := 

\lfj \ G is the Laplacian matrix associated with a subgraph G^^^ G . By counting how 

many edges are present in Gk and how many are absent from Gfc, respectively, the distribution 
of L{k) is computed by 

= :=7ri, i = 1,..., M. (6) 
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When Tk = W{k) inherits the same distribution as L{k) from Gfc. Then, we denote =: 
I 

3.1 Conjunction of Various Consensus Metrics 

When the inter-sampling interval is small enough (to be precise r* < (N — 1)“^), each node 
recursively updates its state as a convex combination of the previous states of its own and its 
neighbors. Every update drives nodes’ states closer to each other and can be thought of as 
attraction of the nodes’ states. Under the independent random network model, we show in 
the following theorem that, as long as G has a directed spanning tree. Algorithm achieves 
consensus, simultaneously in expectation, in mean square, and in almost sure sense. 

Theorem 1 Let Assumptions (Al), (A2), and (A3) hold. Then expectation consensus, mean- 
square consensus, and almost sure consensus are achieved under Algorithm ^ if n £ (O, (N - 

Proof. By Lemma it suffices to show that Algorithm ([^ achieves consensus in expectation. 

Fix a directed spanning tree Gt of graph G and a sampling time tk. Let the root of G^ be 
ii G V, and define a set of nodes := {U}- Denote 

1 ] := (n) A (1 - (N - l)r*). 

Then, there holds g > 0 when r* G (O, (N — 1)“^). We first assume Xi.^{tk) < l/2(xmax(ifc) + 
Xminitk)) while the other case for xq(tfc) > l/2(xmax(^fc) + a^min(ifc)) will be discussed later. 

Ghoose a node i 2 £ V such that i 2 ^ and (iiU 2 ) £ Gt- Define ^2 ■= U {^ 2 }- 
Consider the event (02 := {(^ 1 ,^ 2 ) £ Efc+i}. When N 2 happens, XjjCtfc-i-i) evolves as follows: 

Xi^itk+i) = Wi2i^{k)xi^{tk) + ^ Wi2j{k)xj{tk) 

< ^'>Xi2i,{k){x min (ik) + ^max (^fc)) T (1 (^))®max(^fc) 

— d" (1 ^'n)x.[agji_{tk), 

where the last inequality holds because g < Wi^ij^ik). Since g < Wi.,i.,^{k), we show that xq(tfc+i) 
is bounded by 

Xi^{tk-\-\) ^ T (1 '^'n)Xj]2axitk) ■ 
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At time tk+ 2 , 

Xi^{tk+2) = Wi2i2ik + l)Xi^{tk+l) + ^ Wi^j{k + l)Xj{tk+l) 

j¥=h 

^ ^* 2*2 ~k 1) ~^'n^XYaax{tk) ~l~ ^* 2*2 ~k l))®max(^fc+l) 

— 2^ ^minidk) “1“ (1 2^ )3Jmax(^fe)) 

where the last inequality is due to Xmax(ifc+i) < Xraa.x{tk) and rj < Wi^i^{k + 1). The same is true 
of node ii, i.e., Xi^{tk+ 2 ) < \'rf‘Xmm{tk) + (1 - v‘^)xmax{tk)- Recursively, we see that 

Xii{tk+n) — 2^ 2^min(^fc) T (1 2^ )Xmax{tk) 

and 

Xi 2 {tk+n) — 2 ^ 3^min(^A:) T (1 )®max(^fc)- 

holds for n = 1, 2,.... 

Again, choose a node is G V such that is 0 ^2 and there exists a node j G ^2 satisfying 

(j, is) G Gr- Define ^s := ^ 2 U{is}. Consider the event f^s := {(jCs) ^ Efc _|_2 : (i, is) £ Gr,i G ^ 2 } • 

If f^s happens, we obtain a similar result for node is: 

3:^3 ( 4 + 2 ) < ??(3:ii(4+l) V Xi^{tk+i)) + (1 - ??)Xmax(4+l) 

— 2^ 3:xnin(4) T (h 2 ^ )3:max(4) T (f ^)3:max(4) 

= — 7/ 3;min(4) T (1 2^ )3:max(4)- 

From the same argument as above, 

3^13 (4+n) ^ 2 ^ 3:jnin(4) T (f 2 ^ )3:max(4) 

holds for n = 2, 3,.... 

We choose nodes ii,..., i|\i in sequel and accordingly define ^i,..., and (^ 2 , • • •, <^N- 
Consider (^2; • • •, Ai sequentially happen, then 

3'*m(4+n) ^ 2 ^ 3:min(4) T (1 2 ^ )Xma.x{tk) 

holds for all 1 < m < N and n > N — 1, which entails 

^max (4 +n-i) = maxxi(4+N-i) — rtV ^min (4) + (1 “ 7>V^ ^) ^max (4)- 

I z z 
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In this case, the relationship between X(tfc+N-i) and X{k) is given by 

X(fc + N 1) = 3;max(ifc+N —l) 3;min(tfc-|_|\|_i) 

1 1 

— 2^ ®min(^fc) “1“ (1 —rj )3Jmax(^fc) 

= m- 

If Xi-^{tk) > 1/2 (xmax(ifc) + 3:min(ifc)) IS assumed, a symmetric analysis leads to that, when 
(f2,---,4o sequentially occur, Xmin(tfc+N-i) > Xraa.^{tk) + (1 - \rf~^)xrmn{tk)■ Then 

'N{k + N — 1) is bounded by 


X(/c + N 1) — 3^max(^A:+N —l) ^^niin (^fc+N—1) 

^ 2'max(^A:) Xj^axitk) (1 )3^min(ifc) 

= (l - 2^^"') 


exactly the same result as when Xi^{tk) < l/2(xmax(ifc) + a^min(^fc)) is assumed. Therefore, the 
above inequality holds irrespective of the state of xq(tfc). 

In addition, we know that probability that the events <^ 2 , ■ • • j Ai sequentially occur is 

N 

i=2 

Combining all the above analysis, 

E[^(fc + N - 1)] < «"■' il - El-Um + (1 - «“^‘)E[3r(A:)l 

= ( 1 - 5 ( 91 ?)"-') EMI- (7) 

Since 0 < qr] < 1, then lim^^oo IE[X(/c)] = 0, which completes the proof. □ 

When the inter-sampling interval r* is too large, then W{k) may have negative entries. Con¬ 
sequently, some nodes may repel, so consensus of Algorithm Q may not be achieved. When 
repulsive actions exist, expectation consensus, mean-square consensus, and almost sure consen¬ 
sus are not equivalent in general since the Monotone Convergence Theorem cannot be applied. 
Of course, consensus in mean square still implies expectation consensus as consistent with 
convergence for a sequence of random variables. In the subsequent two subsections, mean-square 
consensus and almost sure consensus/divergence will be separately analyzed. 
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3.2 The Mean-square Consensus Threshold 

In this part, we focus on mean-square consensus. First of all, we give a necessary and sufficient 
mean-square consensus condition in terms of the spectral radius of a matrix that depends on 
r*, G and q, by studying the spectral property of a linear system. Note that the analysis is 
carried out on the spectrum restricted to the smallest invariant subspace containing I — 

The condition is then interpreted as the existence of a critical threshold on the inter-sampling 
intervals, below which Algorithm Q achieves mean-square consensus and above X{k) diverges 
in mean-square sense for some initial state x(to)- This translation relies on the relationship 
between the stability of a certain matrix and the feasibility of a linear matrix inequality. 


Proposition 1 Let Assumptions (Al), (A2), and (A3) hold. Then the following statements are 
equivalent: 


(i) Algorithm Q achieves mean-square consensus; 

(a) There holds /9^E[IT(0) (g) IT(0)](J 0 J)^ < 1, where 




( 8 ) 


(Hi) There exists a matrix 5 > 0 such that 

M 

4,(3) JW^^JSJ{W^^)'J < 5, (9) 

i=l 

where vrj is defined in Q . 


Proof. The proof needs the following lemma. 

Lemma 2 (Lemma 2 in |28| ) For any G G there exist Gi G S", i = 1,2,3,4, such that 

G = {Gi- G 2 ) + (Gs - G4)i 


where i = y/-^. 

Define the difference between the state x{tk) and its average as 

d{k) := x{tk) - ^lTa:(4). 


( 10 ) 
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Evidently, d{k) = Jx{tk)- Since 

N(^k) — 3;jnax(^) 

^ Xmaxitk) ^(^fc) “1“ —1 

< 

= V2||d(fc)|| (11) 

and 

= N-V2||d(/,)||, 

(12) 

linifc^oo IE[X^(A:)] = 0 is equivalent to limfc_^oo IE||(i(A:)|p = 0. From the Cauchy-Schwarz in¬ 
equality, E[di{k)dj{k)] < E[di{k)‘^]^^‘^E[dj{k)‘^]^^‘^ holds for any 1 < i,j < N, which furthermore 
implies the equivalence between lim^^oo E||(i(A:)|p = 0 and lim^^oo E[(i(/c)(i(/c)*] = 0. Thus, to 
study the mean-square consensus, we only need to focus on whether K[d{k)d{k)*] converges to 
a zero matrix. 

Observe that 

d{k) = JW{k - l)x{tk-i) 

= JW{k - l)x{tk-i) - l^JW{k - l)ll'x(4-i) 

= JW{k-l)d{k-l) (13) 

holds for A: = 1, 2,..., where the second equality is due to JW{k)l = J1 = 0. It entails 
E[d{k)diky] = E [JIT(A: - l)d{k - l)dik - iyw{k - 1)'J]. 

Taking vectorization on both sides yields 

vec(E[d(A:)d(A:)*]) = E [{JW{k - 1)) 0 (JIT(A: - l))vec{d{k - l)d{k - 1)*)] 

= E [(JIT(O)) 0 (JIT(O))] vec {E[d{k - l)d{k - 1)*]) 

= (J ® J)E[1T(0) 0 IT(0)] vec {E[d{k - l)d{k - 1)*]) 

= (^{ J <S) J)E[IT(0) ® lT(0)])%ec (d(O)d(O)*) 

= ((J ® J)E[IT(0) ® 1T(0)]) V ® J) vec {x{to)x{toy) 

= (J® J)(e[IT( 0)® 1T(0)](J«) J))%ec(x(to)a;(fo)*), (14) 
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where the first equality is based on the property vec{ABC) = {C' 0A)vec{B) for matrices A, B 
and C of compatible dimensions, and the separation of expectations in the second equality is 
due to the independence of the random interconnections. 

The implications from one statement to the next is provided as follows. 

(i) =► (ii). If p (E[fF(0) ® fT(0)] (J® J)) > 1, there exist a number A with |A| > 1 and a non-zero 
vector V G corresponding to A satisfying K[W (0)® W (0)](J® J)v = Xv. Let vi,... ,vi he all 
the eigenvectors corresponding to the eigenvalue 0 of J® J. Since K[W (0) ® W (0)]( J® J)vi = 0 
for any i = I,... ,1, there holds v ^ £ 1^ and (J ® J)v ^ 0. Therefore 

lim {J 0 J)(e[W{0) ® IT(0)](J® = lim X^{J®J)v / 0. (15) 

k^oo \ / k^oo 


In order to show that mean-square consensus is not achieved for Algorithm Q, it remains to 
prove that v can be expressed as a linear combination of different initial states. Note that there 
exist G G and Gi,..., G 4 G such that v = vec(G) and G = G 2 — G 4 -|- (G 3 — Gi)i by 

Lemma(the order of Gi, G 2 , G 3 and G 4 is immaterial in this lemma). Since each G* can be 
expressed as 

N 

where G* = C/Wdiag{A^*\ ..., A^*^}(t/(*))* with A^*^ G cT(Gi) and =: ..., unitary. 

Then, we have 

4 N 

n = ^ ^ -X^^i\ec{u^\u^-^)*). 

i=i j=i 


By letting x{to) = u^j\ i = 1,...,4 and j = 1,...,N, respectively, we see from (15) that 
mean-square consensus is not achieved for some x{to). 


(ii) {iii). Denote i? := (J® J)E[IT(0) ® IT(0)](J® J). From (ii), 

p{R) = p(e[W{0) ® IF(0)](J ® J)2) 

= ,9(e[1F(0) ® IF(0)](J ® J)) < 1. 


Then, {I — R) ^ exists and is nonsingular, {I — R) ^ given positive definite 

matrix V G there corresponds a nnique matrix S G such that 


vec(I/) = {I — R) vec(S). 


(16) 
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Then, 


vec(y) = [l- E[(JTT(0) J) ® (JTy(O) J)]) vec(5) 
= vec(S' - </>(5)), 


where </>(•) is defined in ([^, which implies S — (t>{S) > 0 by the one-to-one correspondence of the 
vectorization operator. The positive definiteness of S follows from 


vec(S') = (I — R) ^ vec{V) 


OO 

i=0 



implying S = ^ > 0, again by the one-to-one correspondence of the vectorization 

operator. 

(in) =► (z). By the hypothesis, there always exists a fj, G (0,1) satisfying 4>{S) < fiS. Fix 
any given X G and then choose a c > 0 satisfying X < cS. Then, by the linearity and 
non-decreasing properties of 4>{X) in X over the positive semi-definite cone, 

cj)^{X) < 0^(c5) = ccj)’‘{S) < = cia(p^-\S) <■■■< cfi’^S 


holds for all A: G N. It leads to limfc_^oo = 0, which means 

lim i?*^vec(X) = 0. (17) 

k^oo 

In light of Lemma for any G G there exist Xi, X 2 , X 3 , X 4 G S” such that G = 

{Xi — X 2 ) + (X 3 — ^ 4 ) 1 . Then, we see from ( [ItI ) 

lim R^veciG) 
fc—>-oo 

= lim i?^('vec(Xi) — vec(X 2 ) -|- vec(X 3 )i — vec(X 4 )i') 
k^oo \ / 

= 0 . 


Since G is arbitrarily chosen, we have p^E[lF(0) 0 VF(0)](J 0 J)^ = p{R) < 1. Then, 

lim vec(K[d{k)d{k)*]) = (J0 J) lim ("e[1T(0) 0 1F(0)](J0 J)) vec{x{to)x{tQ)*) = 0 
k^oo ^ ' k^oo V / 

holds for any x(to) £ which means limfc_>.oo E[d(/c)d(/c)*] =0. □ 

The following result holds. 
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Theorem 2 Let Assumptions (Al), (A2), and (A3) hold. Then Algorithm 0 achieves mean- 
square consensus if and only ifr^ < where is given by the following quasi-convex optimiza¬ 
tion problem: 


arg min^ — r 


subject to 


JZJ + lV ^{JZ - jl^^Uy) 

* z 


* 


* 


r, z > 0, 

y - rZ > 0, 


^{JZ - JL(^)JF) 
0 


> 0 , 




(18) 

(19) 

( 20 ) 


with *’s standing for entries that are the Hermitian conjugates of entries in the upper triangular 
part. 


Proof. Necessity: Suppose that mean-square consensus is achievable for Algorithm Q, or 
equivalently there exists a matrix 5 > 0 such that 4>{S) < S holds by Proposition First 
we shall show Y)T=i JSJ{W^^y J < JSJ + 11'. Without loss of generality, choose for 

(ui,..., um) an orthonormal basis of with vi = i^l. Then, any vector 0 7 ^ x G M”" can be 
expressed as x = with coefficients ai,..., on not all 0. We have 

N ^ N 

x'(j){S)x= 4>{S)( y^^aiVi') 

i=2 i=2 


and 


x'{.JSJ + 11')x = f s ( -I- al. 


1=2 


i=2 


Since oi,..., on are not all 0 and <f{S) < S, there holds TTiJ5J(VF^*^)'J < JS'J-|-11'. 
Finally, let Z = S and Y = r*S'. By Schur complement lemma, we see that (|18|) and (20) hold. 


In addition, the optimization is a generalized eigenvalue problem, which is quasiconvex 29 


Sufficiency: For any given r* < Xj, there always exist Y and Z such that (18), (19) and (20) 


hold. According to Schur complement lemma, (18) is equivalent to 

M 

JZJ + IT -^TTi{JZ - JL^^JY)Z-^{JZ - JL^^JY)* > 0, 

i=l 
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which gives 

JZJ + 11' > JZJ + '^Tri JL^^JYZ-^YJ{L^^'^)'J - JYJ{L^^)'J - JL^^JYJ 

i=l 
M 

> JZJ + ^7rJnJL«jyj(LW)'j- jyj(L«)'j- 

i=l 

> JZJ - tN^JYJ + tN^c/){Y), 


M 


( 21 ) 


where the second inequality holds by substituting Z~^ with in accordance with (20). 

Therefore, it leads to JYJ + r*ll' > <i){Y). Letting S = JYJ + r*ll', we have 

M 

4>(Y) =j: TTi jiyW j(jyj + nii')j{w^^)'j = (j){S) 

i=l 


and S > 4>{S). In addition, the positive definiteness of S can be seen from the following lemma. 


Lemma 3 There holds JMJ + ell' > 0 for all M > 0 and e > 0, where J is defined in (|^. 

Proof. Choose for (ui,..., un ) an orthonormal basis with vi = 1. For any nonzero vector 

_ N 

N ^ N 

x'{JMJ + ell')x = ^ OLiVj^ M ^ + ea^. 

i=2 i=2 

Since ai,..., om are not all 0 and M > 0, we have x'{JMJ + ell')x >0. □ 

By PropositionAlgorithm ([^ achieves mean-square consensus, which completes the proof. □ 

3.3 Almost Sure Consensus/Divergence 

In this part, we focus on the impact of sampling intervals on almost sure consensus and almost 
sure divergence of Algorithm ([^. The following theorem gives the relationships between r* and 
almost sure consensus/divergence: almost sure divergence is achieved when r* exceeds an upper 
bound and almost sure consensus is guaranteed when r* is sufficiently small. Also note these 
two boundaries are not equal in general. 

Theorem 3 Let Assumptions (Al), (A2), and (A3) hold. 

(i) If <T^ with r* given in Theorem^ Algorithm © achieves almost sure consensus. 
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(a) If Tit > T\, where t\^ G M+ is given by 

2N(r-l) (l-g)log(2N) 


Tj, := inf < r : log 


N - 1 


> 


Q*Q 


, s { t ) > 0 


with g* := min{(l — g)l'^l+l‘^'l : (j, i) g E} and s { t ) := min | A min — 

Algorithm Q diverges almost surely for any initial state x{to) G 

except x{to) _L 1. 


Proof. We start by presenting supporting lemmas. 

Lemma 4 (Lemma (5.6.10) in |30j ) Let A G and e > 0 be given. There is a matrix 

norm || • ||^ such that p{A) < H^Hf < piA) + e. 


Lemma 5 (Borel-Cantelli Lemma) Let {y,S,p) he a probability space. Assume that events 

OO 

£/i G S for all i G N. If Yl < oo, then p{.s/i i.o.) = 0, where “s^i i.o.” means 

j=0 

occurs infinitely often. In addition, assuming that events s/i, i G H, are independent, then 

OO 

p{£^i) = OO implies p {sAi i.o.) = 1. 

i=0 

Proof of (i): Note that 

E[||d(A:)f] =Tr(E[d(A:)d(A:)*]) < ||^ 

The inequality results from the fact that, for any X := [xij] G S” , ||vec(X)|p = Yl=i ^Ij — 
Yl=i^u > = ^(Tr(X))^. Ifr* < n or equivalently p(^E[Ty(0)(8)lT(0)](J(8)J)^ < 1 

by Theorem 2, there exists a matrix norm || • ||j such that ||E[1T(0) (8> 1T(0)](J (8> A < 1 

by Lemma Moreover, by the equivalence of norms on a finite-dimensional vector space, for 
the two norms || • || and || • H.]-, there exists a real number c G M+ implying 


norm, 


1^11 < cIlXl 


for all X G 


From the forgoing observations, (14) and the submultiplicativity of a matrix 


E[\\dm' 


(J <s> J) (e[ 1T(0) ® 1T(0)] (J ® J)) Vec (x(to)a:(to)* 
(e[w(o)®w(o)](j® j)y 


<NV2 
<NV2c 
<NV2c 
<cA^N^/^||vec (x(to)a^(to) 


Ivec {x{tQ)x{to)*) 


E[W(0) ® TT(0)](J ® J) ^ ||vec {x{to)x{toy) 
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Therefore, 

OO 

^E[||(i(fc)||^] < c(l - A)^^N^/^||vec(x(fo)x(fo)*) II < oo, (22) 

k=0 

together with Markov’s inequality resulting in that, 


j;P(||d(A:)|| >d)< {l/5^)Y,n\\d{k)f] < oo 

k=0 k=0 

holds for any J > 0. According to Lemma limfc_^oo ||'^(^)|| = 0 almost surely for any initial 
state x(to) £ Then, P (lim^^oo = 0) = 1 follows from ( [IT| ) and (12). 


Proof of (ii): The rest of the proof consists of three steps. In the first two steps, we construct 
a sequence of i.i.d. random variables and give a lower bound of the averaged rate of divergence 
for this sequence. In the third step, the strong law of large numbers is applied to deduce the 
divergence result. 

Step 1. First of all, observe that for all A: G N and oj G 


\\d{k + l,uj)f =d{k, u)*W{k, uj)'JJW{k, uj)d{k, lo) 


> min 

?;_L1 


v*W{k,u)yJW{k,io)v ||d(A:,a;) 


where the inequality holds because d{k,oj) T 1. If u)'JW{k,u}) + > 1 


for any A; G N and lj G then min||„||=i, 


v*W{k,u}yjW{k,u})v 


with (|11[) and (12) implies that 


P X^{k) > 


X‘^{k-l) 

2N 


= 1 


> 1, which together 


(23) 


holds for all A: G N. Therefore, X{k) > 0 for all A: G N provided that j£(0) > 0. The following 
random variables are well defined: 

X^{k + 1) 




X^{k) ’ 


A: G N. 


One condition guaranteeing Amin(^II^(A:, w)'JIT(A:, w) + > 1 is established as follows. 

Note that for any G 

Amin(bF(A:,cu)'JIT(A:,w) + ill') (24) 

=Amm(r2(i^^*^)'^A^*^ - rJL« - T{L^^y J + 

=rAmin( t(L«)'- JL« - (L«)'j) + 1. (25) 
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Introduce 


T# = inf |r : Amin(r(LW)'- JL® - > 0, VL® E . (26) 


A basic but vital observation is that rj < oo, which makes well dehned. To see this, choose a 
positive number Tj associated with any given E If Amin(^T(T(*))'JLW ——> 
0 for any we are done. Otherwise let v E with ||u|| = 1 be any vector such that 


- JLW - {L^^yj^v < 0. 


(27) 


Using the property Ker((L(*))'JL^®)) = Ker(JL(®)), we deduce from (27) v 0 Ker((L(®))'JLW) 
and V* J V > 0. Let take a new value satisfying 


max||^ll=i u*(JLW + [L^y'J)v v*{JL^^ + {L^y'J)v 

^y\v\\=l,v^-Ker(.jNi))V*{L^y'JLi^V “ ||^||=i,^0K^r(JLW) U*(LW)'JL(®)u 


Then, A min ^r,(L^®^)^ — (L*^®))'J^ > 0. Finally, letting tq = maxi<j<nri, we have 

T"# < To < oo. According to Weyl Theorem (Theorem 4.3.1 in 30 ), A min — 

{L^'-'>y > 0 whenever r > tjj for each E .if. Recalling that L{k,uj) E if, we see that 
r > rj guarantees Amin^IU(^, w)' JIU(A:, w) + ^ 1 for all A: E N and ui E . 

Step 2. First, we propose the following claim. 


Claim. There always exist two (random) nodes i,j E V at each time k such that {j,i) E E 
and \xi{tk) -Xj{tk)\ > 

To prove the claim, fix any time instance k. Without loss of generality, index all the nodes 
in the graph such that Xmin(4) = Xi^{tk) < Xi^{tk) < ■ ■ ■ < Xi^{tk) = Xmax(tfc)- Then, there 
at least exists a node in E {z 2 ,...,*n} satisfying Xi^{tk) — Ti^_i(tfc) > i\r^^(^); otherwise 
Xi^{tk)-Xij^{tk) = Yl^= 2 yiii'tk)-Xii_^{tk)) < T(/c), reaching a contradiction. U\xi{tk)-Xj{tk)\ < 
iq^X(A:) for all {j,i) E E, then neither (i,j) 0 E nor {j,i) 0 E for any i = ii,... ,in-i and 
j = in, ■ ■ ■ ,iN) since Xi^{tk) — Ti„_i(tfe) > iq^X(/c), contradicting with the hypothesis that G 
has a directed spanning tree. 

In view of this claim, for each uj E we choose two nodes ik{uj),jk{<w) E V at time 
k such that (jfc(w),4(a;)) E E and |x4(^)(4) - Xjj^(^)(4)| > i^X(A:,a;). The dependence 
of the node selections on a specific sample path gives rise to a challenge in the subsequent 
analysis. To get rid of this, we introduce an additional sequence of random variables. Let 
be a sequence of i.i.d. random variables dehned on ((0,1)^, (R(0,1))^, [), where B{0, 1) 
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denotes the Borel algebra on (0,1), with Zk{C) = Cfc all C £ (0,1)^ and each Zk uniformly 
distributed in (0,1). Let zo,zi,... and Go,Gi,... be independent. Formally, we are allowed 
to define a product probability space (=5^,5,/x) where =5^ = x (0,1)^, S is the cr-algebra 
generated by {s/ x ■. £ (S(0, 1))^}, and ^ is the probability measure satisfying 

X = P(j 2 /)f (^). Define = cr ((Go, ^^o)) • • • j (Gfci -Sfc))- Introduce a sequence of events 
associated with ik{iw), jk(co} and Zk- 

^{k) = I U^g^N (w X 3^k{i^)) ■■ = {jk{(w)},JCj^(^^){k,uj) = 0| 

with ^k{^) = {C e (0,1)^ : Zk{C) < q*/{l - g)l‘^fcMl+l'^feMl Since ik{u),jk{u:) E Tk-i, one 
can verify &{k) E Sk- If r* > 1, for all {oj,Q) E ^(/c) and /c E N, 


.^(A: + 1,0;) >|xi^(^)(4+i) - Xj^(^)(tfc+i)| 

- X{k,u}). 


={n - l)|x4(^)(tfc) - Xj^(^)(tfc)| 

T* - 1 , 


Direct calculation yields 


N - F 


^((o;,C) E S>{k)) = 


q*q 

i-q- 


Step 3. Now we define random variables 

m{k) = 


if(o;,C)E^(fe), 
2 ^, otherwise; 


which together with (28) leads to 




Therefore, 


which gives 




\k=0 


X^{k + 1) 

X^{k) 


X2(t + 1) 
X2(0) 


> Tf{k) = 1. 


> l[m\k)\ =1, 
k=0 J 


fi (log X(t + 1) — log X(0) > ^ log 9J1(A;) J = 1. 

V k=n J 


(28) 


(29) 


(30) 


(31) 


Since each node samples the neighbors independently, where the “independence” is in both 
spatial and temporal sense (Assumption (A2)), therefore, for any E N, 


/i ((o;, C) E 3!{k) I Sk-i) = = p ((uj, () E ^(k)) 

1 — p 
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indicating that 5Jl(A;)’s are independent random variables for S’{0 ),..., &{k — 1) G Sk-i- By 
induction, we eventually have are i.i.d. with the mean computed as 




(32) 


Additionally, since 9Jl(/c)’s have uniformly bounded covariances, Kolmogorov’s strong law of 
large numbers shows that 


1 


t 


^ = m{T*) = 1, 


(33) 


which together with (31) implies that, when m{T^) > 0, P (liminffc_^oo ^(fc) = oo) = 1. Notice 
that m(r*) is increasing in r*. Defining t\, = inf {r : m(r*) > 0} and choosing r* > rj V r;, := 7^, 
the conclusion follows. □ 


4 Markovian Random Networks 


In this section, we continue to investigate the sampled-data consensus when each node samples 
the neighbors following a Markov chain. The following assumption is imposed. 


(A4) Independently among {j,i) G E, the random variables l{(j,i)eEfc }7 ^ = 0,1,..., are a bi¬ 
nary Markov chain with the failure rate P (l{(j,i)eEj,+i} = 0 | l{(j,i)eEfc} = l) '■= P Ihe 
recovery rate P (l{(j,i)eEfc+i} = 1 I l{(i,i)eEfe} = O) := q positive and strictly less than one. 


Note that the techniques developed in this section also apply when p{i) and q{i) vary depending 
on the node index i. 

Under Assumption (A4), {L{k)}k£f^ are a sequence of random variables taking values from 
.if, governed by a finite-state time-homogeneous Markov chain. The transition probability of 
{L{k)}k^^ is induced from the transition of edges between the “on” state and the “off” state, 
which is 

F(^L{k) = I L{k - 1) = l0)) = /i( 1 -p)"2g"3(l - ■= TTij. (34) 


where si Y2 —oi’’ X) X) and 

/ \ ^T-i l*'Tnn7“^?*'Tnn —/ \ ^-i-i \^mn7~'~''i^mn7~'~'j , \^t-v \^mn — 

(ri.,m)GE (n,m)GE (n,m)GE 

•^4 E ^p^®^ —0 —ou 

(n,m)eE 

For convenience, we denote 11 := [ 77 ,^] as the transition probability matrix of L{k). Again, 
W{k) inherits the same distribution from L{k). The positiveness of the recovery and failure 
rates in Assumption (A4) makes L(k) an ergodic Markov chain and IT a positive matrix. 
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4.1 Conjunction of Various Consensus Metrics 

In this part, we show that an analog of Theorem[^holds over a Markovian random network. From 
the probabilistic point of view, the difference between independent model and Markovian model 
can be interpreted using a finite permutation argument as follows. Let ^ be a finite permutation 
from N onto N such that q{i) ^ i for finitely many i. For any given ui G we define a finite 
permutation as {quj)i = for all i G N. In the i.i.d. model, the probability measure is invariant 
with respect to a finite permutation of the sample path, i.e., P(a; G = P(a; G qJ^); while in 
the Markovian model this property is absent because of the Markov property. Nevertheless, if 
Tfc = T* G (O, (N — 1)“^) for all k, the difference does not play any key role in whether or not 
X{k) converges in expectation for Algorithm Q. Moreover, Lemma guarantees mean-square 
consensus, and almost sure consensus regardless of the random network model. 

Theorem 4 Let Assumptions (Al), (A3), and (A4) hold. Then expectation consensus, mean- 
square consensus, and almost sure consensus are achieved for Algorithm @ ifn G (O, (N-I) ^). 

Proof. The proof is similar to that of Theorem Here we only provide a sketch. Fix a 
directed spanning tree Gt of graph G and a sampling time tk. We choose ii,..., i|\i and define 
... ,.^|\| in sequel by the following iterated algorithm: 1 ) Set ii as the root node of Gt, 
■^1 '■= {H} and I = 2] 2) Choose a node ii G Y such that there exists a node j G 
satisfying (j,q) G Gr and ii 0 3) Update := U {i/}; If / < N, set I = 

I + 1 and go to step S); otherwise stop. Consider a sequence of events <^ 2 , ■ ■ ■ where S'l := 
|l(A: + Z — 1) G G / 0}| for / = 2, 3,..., N. If S’ 2 ,..., Al sequentially occur, 

similar to the proof of Theorem we see that 

X{k + N - 1) < (^1 - X{k) (35) 

where p := (r*) A (1 — (N — l)r*) > 0. Then, we estimate the probability of the sequential 
occurrence of S’ 2 ,..., S '3 by 

P = UL{k-l)GAS) =P(1^, = 1 I = 1) • • = 1 I L(/^ - 1) G 
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where tt := mini<jj<M iTij > 0. Therefore, 
E[X(A;+ N - 1)] 


: E 

7 = 0,1 




.=^\L{k-l))E X(fc + N-1) I 1, 


<(1 - 7r^-i)E[E[T(fe) I L{k - l)]]+7r^-i (^1 - 


= (l--(7rr?)'^-^)E[X(A;)], 


^ - 1 ) 

1 = 2 * J 

E[E[X(A;) I L{k-l)]] 


which implies limfc_,.oo E[3£(A:)] = 0 and therefore consensus in expectation is achieved. Finally, 
the conclusion follows from Lemma [TJ □ 


Remark 3 The assumption of a uniform inter-sampling interval Th simplifies the notations 
used in Theorems [I] and It should be emphasized that the techniques used in the proof of 
Theorems and also apply to the non-uniform inter-sampling interval case. To make the 
conclusion hold, we require limfc_,.oo log (E[X(A:(N —1))]/E[X(0)]) = —oo, which can he guaranteed 
by Er=o n"=-o r]k-\-j = CO withrjk = (rfc)A(l —(N —l)Tfc) fork G N. This is seen from Q and the 
fact that, for a sequence {ak}k&n with ak G [0,1), Ofc = oo z/ and only if n^i(l “ ^k) = 

0 


4.2 The Mean-square Consensus Threshold 

Now, we are interested in establishing a necessary and sufficient condition on r* for mean-square 
consensus of Algorithm Q . We first present an implicit condition in terms of the spectral radius 
of a certain matrix. Then, this stability condition is translated to a threshold on r*. The analysis 
in this section is based on the techniques using in the proof of Proposition as well as the tools 
from the theory of Markov jump linear systems. 

Proposition 2 Let Assumptions (Al), (A3), and (A4) hold and, for each {j,i) G E, l{(j,i)eEo} 
starts at any initial distribution. Then the following statements are equivalent: 

(i) Algorithm Q achieves mean-square consensus; 

(a) There holds /9(r0) < 1, where 

T := diag(^W(i) ® ..., ® (36) 
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and 


0 :=n'® (J® J) 


( 37 ) 


with J defined in and IT defined in (34); 


(in) There exist matrices Si,... ,Sm >0 such that 


M 


TjiS) ■■= Y, 


(38) 


2 = 1 


holds for all 1 < j < M, where S := {Si,..., 5m)- 


Proof. Recall d{k) from (10). Obviously, 0 to ( |13[ ) still hold. In what follows, we consider a 
linear space over the complex field C: := |(Mi,..., Mm) : Mj G i = 1,..., m| and a 

convex cone in := |(Gi,..., Gm) : Gi G S|^, i = 1,..., m|. Define 


Hik) : = 


^{L{k)=LW}’ ■ ■ ■ >'^(^)'^(^)*7{L(fe)=L(M)} j £ 


tM 


Since 


E[d{k)d{kr] = E[H{k)] [/n,...,/nJ \ 

M times 

it follows from 0 and ( [I^ that lim^^oo IE[X^(/i;)] = 0 is equivalent to hmfc_,.oo IE[i7(A:)] = 0. 


Taking vectorization on both side of E[H{k)] gives 

(JIRW) ® (JIRW) 



TTii - 

• ttmi 


vec(E[77(A:)]) = 





Mm • 

• ttmm 



(J1T(^)) ® (J1T(^)) 

vec(E[d(A: - l)d{k - l)*l{Lik-i)=Lm}]) 


Yec{E[d{k - l)d{k - lTl{L{k-i)=Lm}]) 

= ((n' ® /n2)(7m ® (J ® J))r)vec(E[M(A: - 1)]) 

= ((n' ® 42 ) (7m ® (7 ® J))r)%ec (E[M(0)]) 

= (^{Im ® (j ® <7))(n' ® /N2)r) (7 m ® (J ® J)) vec^E [li{x{to), L(0))] ^ 

= (7m ® (7 ® J)) (^(n' ® /M 2 )r(/M ® (7 ® 7))^ vec^E [lt{x{to), 7(0))] ^ , 

where /t(x(to), 7(0)) := (^x{t(i)x{to) *l{L(o)=L(i)}>'-->3;(to)a;(to)*l{L(o)=L(M)}) G BI+- The fourth 
equality holds because {A®B){C®D) = {AC)®{BD) for matrices A, B, C and D of compatible 
dimensions. In addition, p[{P' ® lM 2 )r(/M ® (7 ® 7))^ = /9(r0). 
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It follows from Lemma I that for any H G there exist Hi,..., H4 G IHY' such that 
H = {Hi — H2) + (LI 3 — H^)!. Moreover, for each Hi = ..., 

M N 

m=l n=l 


with Urn =■■ unitary and Gm = C/m^diag{A^|p ..., for m = 

1,..., M, which means that, for any H G vec(i/) can be expressed as a linear combination 
of different initial states. 

The rest of the proof follows from the arguments used in the proof of Theorem and the 


theory of Markov jump linear systems 33 


□ 


The following theorem holds based on Theorem and the theory of Markov jump linear 
systems, so the proof is omitted. 


Theorem 5 Let Assumptions (Al), (A3), and (A 4 ) hold. Then Algorithm Q achieves mean- 
square consensus if and only ifr^, < fi^, where is given by the following quasi-convex optimiza¬ 
tion problem: 


arg mm.,- — r 


subject to 


JZjJ + IT ^/Wf] {JZi - JL^iljYi) 

* Zi 


Yj,Zj > 0 , 

Yj-TZj> 0 , j = 


^TTMjiJZM — JL^^^JYm) 


0 


> 0 , 


Zm 


4.3 Almost Sure Consensus/Divergence 

In this part, we explore the almost sure consensus/divergence condition for Algorithm © over 
Markovian random networks. The following theorem exhibits a correlation between r* and the 
asymptotic behavior of every sample path, that is, a small r* guarantees almost sure consensus 
while a large r* tends to result in almost sure divergence. In the following theorem, the almost 
sure divergence analysis is restricted to complete graphs. The assumption of complete graph 
simplifies the analysis. However, we believe that the techniques used in developing almost sure 
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divergence results in Theorems and can also deal with general directed graphs. We plan to 
remove this restriction and consider more general graphs in future work. 


Theorem 6 Let Assumptions (Al), (A3), and (A 4 ) hold and, for each {j,i) G E, l{(j,i)eEo} 
starts at any initial distribution. 


(i) //r* < fj, then Algorithm (§ achieves almost sure consensus. 


(a) If G is a complete graph and r* > fi,, where ^ is given by 


fi, ;= inf <j r : log ^ > 0, s{t) > 0 , 


with g* := {1 — p) A q and s(t) := min | Amin('r(T(*))' J) : G .i?}, 

then Algorithm Q diverges almost surely for any initial state x{to) G except x( to) T 1. 


Proof. To show (i), note that 


E[||d(fc)f] = TV(E[ij-(fc)] [lN,...,JNj 'j < (MN)V2||vec(E[//(fe)])||. 

M times 

When p(r0) < 1, by using the same argument as in (22), we know that °° 

holds for any initial state x(to) G and any initial distribution of IjQ-for each (j, i) G E. 
By Markov’s inequality and Lemmalim^^oo = 0 almost surely. 


Next, we shall prove (ii). Similar to the proof of Theorem]^ the analysis is divided into 
three steps. 


Step 1. Suppose r > rp, where rjj G M+ is defined in (26). Adopting the analysis used in the 
proof of Theorem we define 


m := 


and conclude that 


holds for all A: G N. 


di\k + l) > 


X^{k + l) 
T2(A:) ’ 

X'^ik) 


2N 


= 1 


(39) 


Step 2. In the first place, for each oj G 5^^, we choose two (random) nodes ik{oj), jk{u}) G V 
at time k such that — a;j^(a;)(Afc)| = X{k,Lo). Let be a sequence of i.i.d. 

random variables defined on ((0,1)^, (.6(0,1))^, C) with Zk{C,) = Cfc all C £ (0,1)^ and each 
Zk uniformly distributed in (0,1), and let zq, zi,... and Go, Gi,... be independent. Formally, 
we are allowed to define a product probability space {y,S,p) with g, the product probability 
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= < 


measure satisfying /i(^ x ^) = for any ^ T and G (^(0,1))^- Define = 

a ((Go, ^^o), ■ • •, (Gfc, Zk))- Introduce a sequence of events associated with ikioj), jk{^) and z^- 

S!{k) = I U^g^N (w X ^fc(w)) : {jk{uj),ik{uj)) G Efcj 

with =^fc(ti;) given by 

{C G (0,1)^ : ZkiC) < {{1-p) Ag)/(1 - p)} , if (jfc(a;), 4(a;)) G Efc_i(a;); 

{C G (0, : ZkiC) < ((1 -p) Cq)/q} , if (jfc(a;),4(a;)) ^Efc_i(a;). 

We have the following claim due to a complete underlying graph G. 

Claim. Suppose r* > 1. There holds X{k + l,u;) > (r* — l)X(A:,a;) for all {ojX) £ ^(k) and 
A: G N. 

Step 3. We define random variables 

9It(A:) = 


if(u;,C)G^(fe); 


2N ’ 


otherwise. 


(40) 


Similar to the proof of Theorem for any t G N, 

/i (log Xit + 1) — log X(0) > ^ log 9JI(/c) J =1. (41) 

V A:=0 / 

Since each node independently samples among its neighbors, for any A: G N, 

{{uJ, C) G 3!{k) I cSfc-i) = (1 - p) A g = /r ((w, C) G &{k)) := g*, 

indicating that 5Jl(A:)’s are independent random variables for S’{0 ),..., ^{k — 1) G Sk-i- By 
induction, we eventually have {9II(A;)}fcgN are i.i.d. with the mean computed as 


E[logfm(A:)] > g* log ^ ^ + (1 - Q*) log ^ := rhin) 


(42) 


In addition, since 5[)I(A:)’s have uniformly bounded covariances, again by Kolmogorov strong law 
of large numbers , 

t 

t—>-00 t 


pL [ lim - log Tlik) = E[log 9II(A:)] | = 1, 

\ f.^na f. Z.—J J 


(43) 


k=0 


together with (31) implying that, when m(r*) > 0, P (liminffc^oo •l£(fe) = oo) = 1. Notice that 
7n(r*) is increasing in r*. The proof is completed by defining f[, = inf {r : m(r*) > 0} and choosing 
T* > Tj V fb := f|,. □ 
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1^2 



Figure 1: A underlying graph G consisting of four nodes. 


5 Numerical Examples 

In this section, we provide numerical examples to validate the theoretical results. We first 
illustrate the existence of the threshold on r*, which decides the mean-square convergence or 
divergence (see Theorems and . We then discuss and illustrate how this threshold depends 
on the number of nodes with cyclic underlying graphs, for i.i.d. and Markovian network models, 
respectively. 


5.1 Mean-square Convergence vs. Divergence 


We consider a network consisting of N = 4 nodes indexed by V = {ui,U 2 ,us,U 4 }. Let E = 
{(^ 1 ,^ 2 ), (^ 2 ,^ 3 ), (u 3 ,U 2 ), ('y 3 ,'r 4 )}. The underlying graph G = (V,E) is illustrated in Eigurej^ 
Evidently, G has a directed spanning tree. The random variables (jg) £ E aiid 

k £ N, are i.i.d. Bernoulli ones with P((j, i) G E^) = 0.5. We choose a uniform inter-sampling 
interval, i.e., = r* for all k. Then Algorithm ^ is given by 

x{tk+i) = [I - nL{k)]x{tk). (44) 


According to Theorem]^ we compute that system (44) achieves consensus in mean square if and 
only if r* < 1.07. We next illustrate this conclusion using simulations. Choose x{to) = [521 1 ]', 
run 10® Monte Carlo simulations, and then use the average as an approximation of E[T^(A:)]. 
Eigure illustrates that E[X^(A:)] converges to 0 as A: becomes large when r* = 1 and diverges 
as k increases when r* = 1.14, validating the conclusion of Theorem 
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Figure 2: The evolutions of E[X^(/c)] for different sample periods over an independent random 
network with q = 0.5. In the upper figure, X‘^{k)] converges to 0 as /c —)■ oo when r* = 1. In the 
bottom figure, X‘^{k)] diverges as /c —)> oo when r* = 1.14. 



Figure 3: An illnstration of a directed cycle graph. 

5.2 Independent and Markovian Random Graphs 

Consider a network of N nodes connected by a directed cycle graph as the nnderlying graph, see 
Figure We choose q = 0.6 for the i.i.d. model. The relationship between the number of nodes 
N and the critical sampling interval is plotted in Figure As for the Markovian model, we 
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choose p = 0.4 and q = 0.7. The relationship between N and is plotted in Figure]^ Note 
that each has a stationary distribution identical to the distribution of l{(j,*)££*} in 

the independent model. 



Figure 4: The relationship between N and over cycle graphs over independent random net¬ 
works {q = 0.6). For N = 3, two sample periods, r* = 1.1 (the red rectangle) and r* = 1.2 
(the blue circle), are chosen to illustrate the divergence and convergence behaviors of E[T^(A:)] 
respectively. 


6 Conclusions 

In this paper, we have considered sampled-data consensus problem over random networks. We 
first defined three types of random consensus notions and established the equivalence of these 
consensus notions provided a sufficient condition in terms of the inter-sampling interval and the 
size of the network. Under this condition, three types of consensus were shown to be simultane¬ 
ously achieved if the underlying graph contains a directed spanning tree. Both independent and 
Markovian random networks are then considered. In either network model, necessary and suffi¬ 
cient conditions for mean-square consensus were derived in terms of the inter-sampling interval. 
Sufficient conditions for almost sure convergence/divergence were also provided, respectively, in 
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Figure 5: The relationship between N and over cycle graphs over Markovian random networks 
{p = 0.6, q = 0.9). For N = 3, two sample periods, r* = 1.0 (the red rectangle) and r* = 1.1 
(the blue circle), are chosen to illustrate the divergence and convergence behaviors of E[X^(A:)] 
respectively. 


terms of the size of the inter-sampling interval. The results for the independent and Markovian 
random networks are summarized in the following table. It is quite surprising that the phase 
transition phenomenon of mean-square consensus exists for both types of random networks. 
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